Repository navigation
test(ci): keep the docs/CI.md check table aligned with workflow jobs - #274
Conversation
|
Thank you, @raymondginger2018-sudo. The job-inventory test is useful, and the first test is good as is. Three changes before merge:
Note that #273 is now on main, so please rebase first. |
Review fixes: parse the Check table's first column instead of searching the whole doc (intro/Rust-formatting false passes), give lint-and-format its own table row, and reword the allowlist comment to release-only so it matches dependency-audit/bundle living in CHECK_TABLE.
12c9862 to
5efea26
Compare
|
@Zongwei9888 — all three changes are pushed as
On the rebase: the branch is already based on the current Local preflight of this exact head (Windows, Python 3.12/3.13/3.14): no new failures; the only red is the already-tracked Windows-only lifecycle flake. |
|
Thank you, @raymondginger2018-sudo! Merged in 3bb8c88. The table-only matching works: I checked that the intro's "formatting" no longer satisfies One follow-up in 49805ef: the new row now names what the check verifies (Ruff lint and formatting, workflow lint, whitespace and file endings) and drops the job id, so it reads like the other rows. The "formatting" phrase still matches its first column. |
Summary
docs/CI.mdexplains each CI check in a table, but nothing stops the table from drifting when a workflow gains, loses, or renames a job. This PR adds a small regression test that parses every workflow file and requires each job to be either mapped to the phrase that documents it indocs/CI.mdor explicitly allowlisted as release-only.Changes
tests/test_ci_documentation.py(new file only):test_every_workflow_job_is_classified— parses.github/workflows/*.{yml,yaml}with pyyaml (already pinned inrequirements.lock;test_ci_installation.pyis a precedent forimport yamlin tests) and asserts the file set and each file's job set matchCHECK_TABLE | RELEASE_ONLY_JOBSexactly: a new workflow file, a new job, or a renamed job fails until classified or allowlisted.test_check_table_phrases_still_appear_in_the_docs— asserts every mapped phrase still exists indocs/CI.md(case-insensitive), so deleting a documented row fails too.RELEASE_ONLY_JOBSallowlists the three release-only workflows (deploy-website.yml,pypi-publish.yml,release-desktop.yml) — jobs that never gate a pull request.encoding="utf-8") so the test does not depend on the runner locale (a plainread_text()dies on Windows cp936 when the docs contain an en-dash).Tests
pytest -q tests/test_ci_documentation.py→ 2 passed (Python 3.12.14, pyyaml 6.0.3).ruff checkandruff format --checkwith the CI-pinned ruff 0.15.21 → clean.KNOWN-RED— verdict linePREFLIGHT: KNOWN-RED | jobs=8 | rc0=14 | new=0 | known=1 | sanitize=clean | tree_head=12c9862e(new=0: no new reds; the single known red is the baselinejob-windows-lifecycle:group2_rc=1, expected on local Windows).notrun=desktop,bundle,web-browsermatches upstream PR scope gating for a tests-only change.Dependency note
docs/CI.mdon main, so the test passes without any docs change; independent of the docs reproduction PR (no file overlap).AI-Assisted Disclosure